Optimal robust estimates using the Kullback-Leibler divergence

Hampel (1974) introduced a very general procedure to derive optimal robust M-estimates for oneparameter families of distributions. The optimal estimate is obtained by minimizing the asymptotic variance among M-estimates which are Fisher-consistent and have gross error sensitivity (GES) bounded by a given constant. Stahel (1981) generalized the optimal M-estimates for the case of families which depend on several parameters. Let f(x, θ) be a family of densities, where x ∈ R and θ ∈ Θ ⊂ R and let Fθ be the corresponding distribution functions. An estimating functional of θ is a function T : Fp → Θ, where Fp is the space of distribution functions on R. The influence function of T is defined as IF (T, x, θ) = ∂T (Fθ,x,ε) ∂ε ∣∣∣∣ ε=0 , where Fθ,x,ε = (1− ε)Fθ + εδx and δx is the point mass distribution at x. Let x1, ..., xn be a random sample of Fθ and let θ̂ n = T (Fn), where Fn is the empirical distribution. Under general regularity conditions we have n(θ̂ n − θ)→D N(0, VT (θ)), (1) where →D denotes convergence in distribution and VT (θ) = Eθ(IF (T, x, θ)IF (T, x, θ)′), (2) where ′ denotes the transpose and Eθ is the expectation with respect to Fθ. The optimal estimates proposed by Stahel are defined by minimizing the trace of VT (θ) among the class of Fisherconsistent M-estimates with GES bounded by a given constant. However these estimates depend on the way in which GES is defined in the multiparameter case. One possibility is to consider the unstandardized GES γu(T, θ) = sup x ||IF (T, x, θ)||. This definition in not invariant under model reparameterizations, and therefore the corresponding optimal estimate is not equivariant. Stahel (1981) proposed two invariant definitions of GES. The first is the self-standardized GES, which is standardized using its own asymptotic covariance matrix VT (θ) and the second invariant definition of GES is standardized using the information matrix. Stahel (1981) derived the optimal M-estimates for the non invariant and the two invariant GES definitions. However, it is not always clear which is the most appropiate standardization for a particular application. These results can be found also in Chapter 4 of Hampel et al. (1986). To overcome this problem, in this paper we propose optimal M-estimates which use measures of robustness and efficiency based on the Kullback-Leibler (KL) divergence. The advantage of this approach is that these measures do not depend on the particular parametrization of the family of distributions, and as a consequence, the resulting optimal estimates are equivariant. We show that the optimal estimates obtained using this approach coincide with the optimal M-estimates obtained when the GES is standardized using the information matrix. 2 Robust estimates using the Kullback-Leibler divergence